为基于几何的点云压缩(G-PCC)标准开发了基于学习的自适应环滤波器,以减少属性压缩工件。提出的方法首先生成多个最可行的样品偏移(MPSO)作为潜在的压缩失真近似值,然后线性权重以减轻伪影。因此,我们将过滤后的重建驱动尽可能靠近未压缩的PCA。为此,我们设计了一个由两个连续的处理阶段组成的压缩工件还原网络(CARNET):MPSOS推导和MPSOS组合。 MPSOS派生使用两个流网络来模拟来自直接空间嵌入和频率依赖性嵌入的局部邻域变化,在该嵌入中,稀疏的卷积被利用可从细微和不规则分布的点中最佳汇总信息。 MPSOS组合由最小平方误量学指导,以进一步捕获输入PCAS的内容动力学,从而得出加权系数。 Carnet作为GPCC的环内过滤工具实现,其中这些线性加权系数被封装在比特斯流中,并以忽略不计的比特率开销。实验结果表明,对最新的GPCC的主观和客观性都显着改善。
translated by 谷歌翻译
本研究通过基于稀疏的张量处理(STP)的Voxelized PCG的多尺度表示,通过稀疏的张解器处理(STP)进行了一种统一点云几何形状(PCG)压缩方法。应用STP显着降低了复杂性,因为它只执行以最可能的积极占用体素(MP-POV)为中心的卷曲。并且多尺度代表有助于我们逐步压缩规模明智的MP-POV。总压缩效率高度取决于每个MP-POV的占用概率的近似精度。因此,我们设计基于稀疏的卷积的神经网络(Sparsecnn),包括稀疏卷曲和体素重新采样以广泛利用前沿。然后,我们开发基于SPARSECNN的占用概率近似(SOPA)模型,以估计在单阶段的方式中仅在逐步使用自回归邻居之前或以多阶段使用的横级或以多级的方式估计占用概率。此外,我们还建议基于SPARSECNN的本地邻居嵌入(SLNE),以表征当地空间变化作为改进SOPA的特征属性。我们的统一方法显示了在与MPEG G-PCC相比的各种数据集中,包括致密PCG(8iVFB,OWLII)和稀疏LIDAR PCG(KITTI,FORD)的各种数据集中的无损压缩模式中的最先进的性能和其他基于学习的压缩方案。此外,所提出的方法由于跨越所有尺度的模型共享而引起的轻量级复杂性,并且由于模型共享。我们使所有材料可在HTTPS://github.com/njuvision/sparsepcgc上公开访问可重复的研究。
translated by 谷歌翻译
本文提出了解码器 - 侧交叉分辨率合成(CRS)模块,以追求更好的压缩效率超出最新的通用视频编码(VVC),在那里我们在原始高分辨率(HR)处编码帧内帧,以较低的分辨率压缩帧帧间( LR),然后通过在先前的HR帧内和相邻的LR帧间帧内解解码LR帧间帧间帧帧。对于LR帧间帧,设计运动对准和聚合网络(MAN)以产生时间汇总的运动表示,以最佳保证时间平滑度;使用另一个纹理补偿网络(TCN)来生成从解码的HR帧内帧的纹理表示,以便更好地增强空间细节;最后,相似性驱动的融合引擎将运动和纹理表示合成为Upscale LR帧帧,以便去除压缩和分辨率重新采样噪声。我们使用所提出的CRS增强VVC,显示平均为8.76%和11.93%BJ {\ O} NTEGAARD Delta率(BD速率)分别在随机接入(RA)和低延延迟P(LDP)设置中的最新VVC锚点。此外,对基于最先进的超分辨率(SR)的VVC增强方法和消融研究的实验比较,进一步报告了所提出的算法的卓越效率和泛化。所有材料都将在HTTPS://njuvision.github.io /crs上公开进行可重复的研究。
translated by 谷歌翻译
Different people speak with diverse personalized speaking styles. Although existing one-shot talking head methods have made significant progress in lip sync, natural facial expressions, and stable head motions, they still cannot generate diverse speaking styles in the final talking head videos. To tackle this problem, we propose a one-shot style-controllable talking face generation framework. In a nutshell, we aim to attain a speaking style from an arbitrary reference speaking video and then drive the one-shot portrait to speak with the reference speaking style and another piece of audio. Specifically, we first develop a style encoder to extract dynamic facial motion patterns of a style reference video and then encode them into a style code. Afterward, we introduce a style-controllable decoder to synthesize stylized facial animations from the speech content and style code. In order to integrate the reference speaking style into generated videos, we design a style-aware adaptive transformer, which enables the encoded style code to adjust the weights of the feed-forward layers accordingly. Thanks to the style-aware adaptation mechanism, the reference speaking style can be better embedded into synthesized videos during decoding. Extensive experiments demonstrate that our method is capable of generating talking head videos with diverse speaking styles from only one portrait image and an audio clip while achieving authentic visual effects. Project Page: https://github.com/FuxiVirtualHuman/styletalk.
translated by 谷歌翻译
Unmanned aerial vehicle (UAV) swarms are considered as a promising technique for next-generation communication networks due to their flexibility, mobility, low cost, and the ability to collaboratively and autonomously provide services. Distributed learning (DL) enables UAV swarms to intelligently provide communication services, multi-directional remote surveillance, and target tracking. In this survey, we first introduce several popular DL algorithms such as federated learning (FL), multi-agent Reinforcement Learning (MARL), distributed inference, and split learning, and present a comprehensive overview of their applications for UAV swarms, such as trajectory design, power control, wireless resource allocation, user assignment, perception, and satellite communications. Then, we present several state-of-the-art applications of UAV swarms in wireless communication systems, such us reconfigurable intelligent surface (RIS), virtual reality (VR), semantic communications, and discuss the problems and challenges that DL-enabled UAV swarms can solve in these applications. Finally, we describe open problems of using DL in UAV swarms and future research directions of DL enabled UAV swarms. In summary, this survey provides a comprehensive survey of various DL applications for UAV swarms in extensive scenarios.
translated by 谷歌翻译
In this work, we focus on instance-level open vocabulary segmentation, intending to expand a segmenter for instance-wise novel categories without mask annotations. We investigate a simple yet effective framework with the help of image captions, focusing on exploiting thousands of object nouns in captions to discover instances of novel classes. Rather than adopting pretrained caption models or using massive caption datasets with complex pipelines, we propose an end-to-end solution from two aspects: caption grounding and caption generation. In particular, we devise a joint Caption Grounding and Generation (CGG) framework based on a Mask Transformer baseline. The framework has a novel grounding loss that performs explicit and implicit multi-modal feature alignments. We further design a lightweight caption generation head to allow for additional caption supervision. We find that grounding and generation complement each other, significantly enhancing the segmentation performance for novel categories. We conduct extensive experiments on the COCO dataset with two settings: Open Vocabulary Instance Segmentation (OVIS) and Open Set Panoptic Segmentation (OSPS). The results demonstrate the superiority of our CGG framework over previous OVIS methods, achieving a large improvement of 6.8% mAP on novel classes without extra caption data. Our method also achieves over 15% PQ improvements for novel classes on the OSPS benchmark under various settings.
translated by 谷歌翻译
Non-line-of-sight (NLOS) imaging aims to reconstruct the three-dimensional hidden scenes from the data measured in the line-of-sight, which uses photon time-of-flight information encoded in light after multiple diffuse reflections. The under-sampled scanning data can facilitate fast imaging. However, the resulting reconstruction problem becomes a serious ill-posed inverse problem, the solution of which is of high possibility to be degraded due to noises and distortions. In this paper, we propose two novel NLOS reconstruction models based on curvature regularization, i.e., the object-domain curvature regularization model and the dual (i.e., signal and object)-domain curvature regularization model. Fast numerical optimization algorithms are developed relying on the alternating direction method of multipliers (ADMM) with the backtracking stepsize rule, which are further accelerated by GPU implementation. We evaluate the proposed algorithms on both synthetic and real datasets, which achieve state-of-the-art performance, especially in the compressed sensing setting. All our codes and data are available at https://github.com/Duanlab123/CurvNLOS.
translated by 谷歌翻译
Stance detection refers to the task of extracting the standpoint (Favor, Against or Neither) towards a target in given texts. Such research gains increasing attention with the proliferation of social media contents. The conventional framework of handling stance detection is converting it into text classification tasks. Deep learning models have already replaced rule-based models and traditional machine learning models in solving such problems. Current deep neural networks are facing two main challenges which are insufficient labeled data and information in social media posts and the unexplainable nature of deep learning models. A new pre-trained language model chatGPT was launched on Nov 30, 2022. For the stance detection tasks, our experiments show that ChatGPT can achieve SOTA or similar performance for commonly used datasets including SemEval-2016 and P-Stance. At the same time, ChatGPT can provide explanation for its own prediction, which is beyond the capability of any existing model. The explanations for the cases it cannot provide classification results are especially useful. ChatGPT has the potential to be the best AI model for stance detection tasks in NLP, or at least change the research paradigm of this field. ChatGPT also opens up the possibility of building explanatory AI for stance detection.
translated by 谷歌翻译
Motivated by the problem of matching vertices in two correlated Erd\H{o}s-R\'enyi graphs, we study the problem of matching two correlated Gaussian Wigner matrices. We propose an iterative matching algorithm, which succeeds in polynomial time as long as the correlation between the two Gaussian matrices does not vanish. Our result is the first polynomial time algorithm that solves a graph matching type of problem when the correlation is an arbitrarily small constant.
translated by 谷歌翻译
Existing solutions to network scheduling typically assume that the instantaneous link rates are completely known before a scheduling decision is made or consider a bandit setting where the accurate link quality is discovered only after it has been used for data transmission. In practice, the decision maker can obtain (relatively accurate) channel information, e.g., through beamforming in mmWave networks, right before data transmission. However, frequent beamforming incurs a formidable overhead in densely deployed mmWave WLANs. In this paper, we consider the important problem of throughput optimization with joint link probing and scheduling. The problem is challenging even when the link rate distributions are pre-known (the offline setting) due to the necessity of balancing the information gains from probing and the cost of reducing the data transmission opportunity. We develop an approximation algorithm with guaranteed performance when the probing decision is non-adaptive, and a dynamic programming based solution for the more challenging adaptive setting. We further extend our solutions to the online setting with unknown link rate distributions and develop a contextual-bandit based algorithm and derive its regret bound. Numerical results using data traces collected from real-world mmWave deployments demonstrate the efficiency of our solutions.
translated by 谷歌翻译